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ABSTRACT 


Failure  discounting  is  the  practice  of  removing  fractions  of  failures 
from  test  data  after  corrective  actions  have  been  taken  and  no  failures 
due  to  the  same  cause  have  reoccurred.  This  thesis  examines  the  effect 
of  discounting  failures  and  weighting  test  data  on  the  accuracy  of  an 
existing  reliability  growth  model,  labeled  the  Modified  AMSAA  model. 
Computer  simulation  is  used  to  evaluate  the  mean  and  mean  square  error 
of  failure  rate  estimates  under  the  model  for  a  variety  of  reliability 
growth  patterns  each  with  several  discounting  and  weighting  scenarios. 
Exponential  failure  times  are  assumed  and  testing  is  truncated  at  two 
failures  in  each  test  phase.  Failure  discounting  tended  to  decrease  the 
mean  square  error  slightly  for  growth  patterns  with  a  continual  drop  in 
failure  rate  for  each  new  test  phase,  but  tended  to  increase  the  mean 
square  error  for  other  patterns.  The  Modified  AMSAA  model  is  also  shown 
to  be  superior  to  the  standard  AMSAA  reliability  growth  model  in  bias  and 
mean  square  error.  No  discernable  benefits  due  to  weighting  the  data  were 
detected. 
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THESIS  DISCLAIMER 


The  reader  is  cautioned  that  computer  programs  developed  in  this  research 
may  not  have  been  exercised  for  all  cases  of  interest.  While  every  effort 
has  been  made,  with  time  available,  to  ensure  that  the  programs  are  free 
of  computational  and  logic  errors,  they  cannot  be  considered  validated. 
Any  application  of  these  programs  without  additional  verification  is  at 
the  risk  of  the  user 
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I.  BACKGROUND 


The  AMSAA  model  is  a  widely  used  cumulative  reliability  growth 
model.  It  was  developed  by  the  Army  Material  Systems  Analysis  Activity 
(AMSAA)  at  Aberdeen  Proving  Grounds,  Maryland.  Failure  discounting 
applied  to  this  and  other  reliability  growth  models  is  becoming  popular. 
In  previous  work  by  Woods  [ref.  1)  the  Modified  AMSAA  model  appears  to 
provide  more  accurate  estimates  of  current  reliability  for  various 
patterns  of  reliability  growth  than  does  the  standard  AMSAA  model. 

The  primary  purpose  of  this  thesis  is  to  determine  the  effect  of 
failure  discounting  and  data  weighting  on  the  Modified  AMSAA  model. 
Secondary  purposes  are  to  compare  the  accuracies  of  the  AMSAA  model  and 
the  Modified  AMSAA  models.  These  comparisons  are  made  for  a  variety  of 
reliability  growth  patterns,  failure  discounting  rates,  and  weighting 
scenarios. 

Throughout  this  thesis  the  phrase  'test  phase'  refers  to  a  collection 
of  tests  wherein  the  nontruncated  failure  times  of  all  items  tested  are 
independent  and  exponentially  distributed  with  a  common  failure  rate. 
Within  each  phase,  n  items  are  tested  until  r  fail,  and  testing  is 
performed  sequentially  by  phase.  Testing  in  phase  i+1  is  initiated  after 
the  failures  in  phase  i  have  been  analyzed  and  appropriate  changes  have 
been  made.  Both  n  and  r  are  input  parameters  to  the  simulation  program 
used  to  generate  test  data. 
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In  application  of  the  Modified  AMSAA  model,  if  the  data  has  a  Weibull 
distribution  with  shape  parameter,  /5,  then  raising  the  data  to  the  ^  power 
yields  exponential  data.  Applying  the  model  to  this  transformed  data 
should  yield  estimates  as  accurate  as  those  indicated  in  this  thesis. 
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II.  THE  AHSAA  RELIABILITY  GROWTH  MODEL 


The  AMSAA  reliability  growth  model  has  its  roots  in  a  report  by  J.  T. 
Duane  of  General  Electric  Company  published  in  1962.  In  this  report  he 
presented  observations  on  failure  data  for  five  types  of  systems  during 
their  development  programs  at  General  Electric.  His  analysis  revealed 
that  the  observed  cumulative  failure  rate  versus  the  cumulative  operating 
hours  fell  close  to  a  straight  line  when  plotted  on  log-log  paper.  That 
is,  if  N(t)  denotes  the  total  number  of  failures  observed  in  t  hours  of 
testing  and  C(t)  is  the  cumulative  failure  rate  observed  over  that  same 
time,  then  log  C{t)  is  almost  a  linear  function  of  log  t  [ref.  2:  p.29]. 
The  model  which  Duane  used  to  interpret  those  plots  has  a  cumulative 
failure  rate  estimate  C(t)  with  expectation  given  by 

E[C(t)]  =  E[N(t)/t]  =  Xt^-'  (2.1). 

Letting  X  =  b  and  =  1-a  results  in 

E[C(t)]  =  bt*^  (2.2). 

Duane  defined  the  instantaneous  failure  rate  r(t)  by  d/(E[N(t)])/dt. 
Applying  this  to  E[N(t)]  =  Xt^  and  differentiating  yields 

r(t)  =  X^t®'^  =  {l-a)bf*  (2.3), 

where  t  denotes  total  accumulated  test  time  [ref.  2:  p.29]. 

The  expression  X^t^'^  is  the  failure  rate  function  for  the  Weibull 
distribution.  This  does  not  mean,  however,  that  the  failure  times  have 
a  Weibull  distribution,  because  the  underlying  distribution  of  the  failure 
times  changes  each  time  a  change  is  made  in  the  hardware.  It  only  means 


that  the  failure  rate  estimate  r(t),  as  a  function  of  the  cumulative  test 
time,  t,  is  changing  in  accordance  with  the  expression  in  in  equation 
(2.3). 

This  model  mixes  data  from  different  populations  to  obtain  an 
estimate  of  current  reliability.  This  apparent  weakness  motivated  the 
development  of  the  Modified  AMSAA  model.  The  unmodified  AMSAA  model  is 
also  referred  to  as  the  cumulative  AMSAA  model  in  some  graphs  that  follow. 

"The  AMSAA  Reliability  Growth  Model  assumes  that  system  failures 
during  a  development  testing  phase  follow  the  (nonhomogeneous)  Poisson 
Process  with  Weibull  intensity  r(t)  =  ,  where  X  >  0,  >  0."  [ref. 
2:  p.  29]  The  parameters  are  estimated  in  the  following  manner  where  N 
=  the  number  of  failures  and  X,  =  total  test  time  up  to  i^^  failure. 


r(t)  =  X0t  =  (l-a)bt 

(2.4) 

A  A  M 

1-a  =  ^  =  - - - 

iog(x,/x,) 

(2.5) 

A  A  M 

b  =  X - ^ 

(2.6) 

r{t)  =  =  {l-a)bf^ 

(2.7). 

Note  that: 

r(T)  =  ^ —  (2.8), 

T^  T 

which  "...is  equivalent  to  using  the  exponential  method  but  purging 
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(l-^)N  failures"  [ref.  2:  p.SO].  This  indicates  that  the  model  is  self 
purging,  and  no  further  failure  removal  method  is  required.  Some  people 
have  developed  failure  discounting  methods  for  this  model. 
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III.  THE  HODIFIED  AHSAA  MODEL 

The  Modified  AMSAA  Model  uses  exponential  regression  to  compute 
estimates  for  a  and  b  after  each  phase.  Test  time  within  each  phase  is 
treated  as  an  independent  observation,  and  the  estimator  for  failure  rate 
within  each  phase,  given  by  equation  (3.1),  is  used  as  the  dependent 
variable  in  the  regression  analysis.  The  cumulative  AMSAA  model  requires 
one  estimate  of  a  and  b  to  cover  all  phases. 

The  Modified  AMSAA  model  uses  the  same  expression  for  current  failure 
rate  as  given  in  the  AMSAA  model,  namely,  =  (l-a)b(TTj)'®, where  TT^  is 
the  total  accumulated  test  time  over  all  phases  at  the  end  of  phase  j 
[ref.  1:  p.  3-1].  Within  each  test  phase  j,  a  failure  rate  estimate  is 
computed  by 

2Fj-l 

j  j 

ifFj<l.  (3.1) 

j 

where  Fj  is  the  number  of  observed  failures  in  phase  j,  and  T^  is  the  total 
accumulated  test  time  in  phase  j.  Explanation  for  the  bias  correction, 
(2Fj-l)/2Fj,  is  developed  in  Appendix  A. 

Let  j  denote  the  current  phase  of  testing  just  completed,  and  let 

TT:  =  2  T.  Y,  =  In  A,,  X,  =  In  TT  ,  for  i  =  1,2, ...,j.  Let 

k  =  l  ■ 
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Yj  =  (Y1+Y2+. .  .+Yj)/j  and  =  {XJ+X2+. .  .+Xj)/j .  The  current  failure 

A 

rate  estimate,  is  given  by 

=  (l-a)b(TTj)-^  (3.2). 

A  A 

Equations  for  aj  and  b^  are  given  by 
,  I  X,Y,  -  Y,  2  X, 

a  .  -  (3.3) 

-  j  J  , 

Xj  2  X,  -  2  X,' 

i=l  i=l 

bj  -  -h:—  exp(Vj+  ajXj)  (3.4) 

for  j  =  2,3, _  The  regression  requires  observations  from  at  least  two 

test  phases.  The  instantaneous  failure  rate  estimate  given  by  equation 
(3.1)  for  j=l  is  used  for  the  first  phase  [ref.  1:  p.  3-2].  Equations 
(3.3)  and  (3.4)  are  developed  in  appendix  A. 

A.  WEIGHTING 

This  model  can  be  modified  further  by  the  application  of  regression 
weights  in  an  attempt  to  make  it  more  sensitive  to  changes  in  failure  rate 
between  phases.  Weighted  regression  is  generally  used  when  some 
observations  are  less  reliable  than  others.  Generally  this  implies 
unequal  variance  among  the  observations  [ref.  3:  p.  77].  To  accommodate 
this  phenomena,  weights  used  in  weighted  regression  are  heavier  for  phases 
with  lower  variance  or  more  reliable  observations.  We  can  use  a  weighting 
scheme  based  on  estimated  variance  or  just  use  a  set  of  weights  chosen  by 
some  other  scheme. 
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The  principle  of  weighted  least  squares  regression  is  to  minimize 
the  weighted  sum  of  squared  differences  between  observed  values  and 
predicted  values,  2  w^  (Y^  -  a  -  ^  In  weighted  regression  the  normal 
least  squares  regression  equations  become 


A  i?i(Xi  -  XJ  Y,w, 

p  -  - 

,i(Xi  -  Kf  w, 


I 

(3.5)  and 


a  =  Y,  -  Mw  (3.6) 

where  and  Y^  are  the  weighted  averages  of  the  data  points,  and  the 
linear  relationship  between  X^  and  Y^  is  described  by  the  model  Y^  =  a  + 
^  X^  [ref.  4:  p.  89]. 

In  this  thesis,  two  weighting  procedures  are  analyzed.  In  method 
one,  the  weights  are  calculated  within  the  program.  In  this  case  the 
quantities  w/  =  f^^’  are  computed  where  f  is  a  number  between  0  and  I 
chosen  by  the  user  as  a  parameter  for  the  program.  These  quantities  are 
normalized  to  yield  the  weights.  With  this  procedure,  the  data  from  the 
current  phase  is  always  given  the  most  weight  when  the  instantaneous 
failure  rate  is  calculated  for  that  phase.  The  smaller  the  value  of  f, 
the  greater  the  effect  of  weighting. 

When  the  user  chooses  the  weights  (method  two),  the  chosen  values  for 
each  phase,  w,  ,  are  read  in  from  a  data  file.  In  both  methods  the  w^* 
values  are  normalized  to  obtain  weights,  w^,  to  use  in  the  regression 
equations.  The  resulting  weights  are  given  by 
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* 

w 

W,  =  — ^ -  ,  i  =  1,2, ...,j  (3.7). 

^  * 

2  w. 

lc=l 

When  using  regression  weights  with  the  modified  AMSAA  model, 

A 

the  data  pairs  (In  In  TT,)  are  used  in  the  weighted  regression 

A 

equations  as  follows:  *  In  X^,  =  In  TTp 

Ywj  =  (WiYi+W2Y2+...+WjYj)/j,  and  -  {W1X1+W2X2+. .  .+WjXj)/j 
for  i  =  1,2, ...  ,j  and  j  =  1,2, - 

-  2  (X,  -  X,j)  Y,  w, 

-  (3.8) 

2  (X,  -  X,p2 

i  =  l 

bj  =  — A —  exp(Y„j+  ajXWj)  (3.9) 

for  j  =  2,3, _  As  in  the  previous  model,  the  instantaneous  failure  rate 

estimate  given  by  equation  3-1  or  3-2  for  j=l  is  used  for  the  first  phase. 

B.  FAILURE  DISCOUNTING 

Another  modification  that  can  be  applied  to  this  model  is  fractional 

failure  removal  known  as  failure  discounting.  Some  investigations  have 

been  made  on  the  application  of  failure  discounting  to  discrete 

reliability  growth  models  [refs.  5  and  6].  What  follows  here  is  a  brief 

description  of  failure  discounting  from  previous  work. 

Testing  conducted  during  the  initial  stages  of  a  particular  system 
often  indicates  a  low  reliability.  Generally,  weaknesses  in  the 
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configuration  of  the  system  or  defects  in  the  quality  of  its 
components  cause  system  failure.  Test  designs  are  established  so 
that  the  cause  for  these  failures  can  be  identified  and  corrected. 
Theoretically,  then,  as  a  weakness  or  a  defect  is  identified  and, 
hopefully,  corrected  the  probability  of  that  particular  weakness  or 
defect  reoccurring  should  be  reduced.  This  reduction  in  the 
probability  of  occurrence  of  a  certain  failure  cause  leads  to 
improved  system  reliability.  This  concept  is  fully  utilized  in 
failure  discounting. 

In  order  to  effectively  discount  previous  failures  it  is  critical 
that  the  cause  of  the  failure  be  properly  identified.  The  level  of 
detail  that  one  wishes  to  ascribe  to  this  identification  process  is 
dependent  upon  the  type  of  system  being  evaluated  and  the  purpose  of 
the  test.  If  a  complex  system  is  being  evaluated  then  a  failure 
cause  may  be  failure  of  a  certain  component  or  sub  component.  The 
precise  element  that  caused  system  failure  is  not  critical  but  the 
ability  to  assign  a  failure  cause  to  each  system  failure  is 
[critical].  Correctly  determining  failure  cause  is  very  difficult, 
particularly  when  dealing  with  complex  systems.  Therefore,  it  is 
conceivable  that  the  design  changes  do  not  improve  system 
reliability.  In  fact,  these  changes  may  even  degrade  reliability. 

To  apply  failure  discounting  procedures  described  below,  one  must  be 
able  to  assign  failure  causes  to  every  system  failure...  [ref.  5: 
pp.  3-4]. 

In  this  model  the  number  of  failures  used  in  equation  (3.1)  is 
adjusted  by  removing  a  fixed  fraction  of  a  failure  each  time  a 
predetermined  amount  of  test  time  has  past  without  reoccurrence  of  the 
failure  cause.  To  employ  the  standard  or  straight  percent  discounting 
method  one  must  specify  two  parameters.  These  are  the  fraction  of  a 
failure  to  be  removed,  f,  and  the  discount  interval,  Treq.  The  adjusted 
number  of  failures  due  to  a  single  cause  is  calculated  as 

Fadj  *  F  .  (l.f)INT(Tsf/Treq)  (3.10), 

where  Tsf  »  Time  Since  Last  Failure  for  the  cause  and  where  INT(x)  is  a 
function  that  returns  the  integer-part  of  x  by  truncating  any  fractional 
part.  When  a  failure  due  to  the  same  cause  reoccurs,  Tsf  is  reset  to  0, 
and  any  previously  removed  fractions  are  restored. 
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A  new  number  of  failures  for  each  phase  is  calculated  as  the  sum  of 
the  adjusted  failures  for  each  cause  in  that  phase.  These  adjusted 
numbers  of  failures  are  used  in  equation  (3.1)  and  the  results  used  in 


equations  (3.3)  and  (3.4)  as  described  above. 

An  example  of  the  application  of  the  standard  method  of  failure 
discounting  will  clarify  this  process.  Consider  the  data  in  Table  I. 
This  fictitious  data  is  intended  solely  to  illustrate  this  discounting 

Table  I;  STANDARD  METHOD  OF  FAILURE  D1SC0UNT1N6 _ 

PHASE  TIME  ACTUAL  FAILURES  CAUSE  ADJUSTED  FAILURES 

12  1  X  1.00 

232  Y  0.50  +  1.00  =  1.50 

373  X  2.00  +  0.25  =  2.25 

f  =  0.5,  Treq  =  3 


method.  Assume  that  improvements  are  made  after  each  failure.  Thus  Table 
I  represents  three  phases  of  a  test-fix-test  scenario  with  two  possible 
causes  of  failure,  X  and  Y.  In  this  example  Treq  =  3  mission  units  and 
f  »  .5 

The  second  failure,  attributed  to  cause  Y,  terminates  phase  two.  At 
that  time  the  failure  due  to  cause  X  has  had  3  mission  units  without  a 
reoccurrence  of  that  cause  so  the  failure  discounting  formula  is  applied: 
^adj  “  F(l-f)"*^‘^*^^^"*‘’^  -  1  (1-0.5)'"^^^^^^  =  0.50 

Thus  at  the  end  of  phase  two  the  failure  that  ended  phase  one  is  counted 
as  one-half  of  a  failure.  Phase  three  ends  with  a  reoccurrence  of  failure 
cause  X.  The  value  for  Tsf  is  reset  to  0  resulting  in  the  restoring  of 
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all  failures  due  to  cause  X  to  full  value.  However  there  have  been  7 
mission  units  since  a  failure  due  to  cause  Y,  Therefore  the  discounting 
formula  is  applied: 

^dj  =  =  (0.50)^  =  0.25 
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IV.  SIHULATION  HETHODOLOGY 


Monte  Carlo  simulation  methods  are  used  to  analyze  the  models 
discussed  in  this  thesis.  The  simulation  is  written  for  a  micro-computer 
in  the  programming  language  Fortran  (see  appendix  B  for  listing).  A 
Monte-Carlo  Simulation  was  used  for  several  reasons.  First,  the  data  can 
be  generated  from  a  known  distribution;  in  this  case  exponential  failure 
times.  Second,  the  parameters,  i.e.  failure  rate,  can  be  controlled. 
Third,  the  results  are  reproducible  if  the  same  seed  is  used.  This  allows 
the  parameters  for  the  failure  discounting  method  and  regression  weights 
to  be  altered  and  compared  using  the  same  data.  Finally  the  data  is  very 
easy  and  inexpensive  to  generate,  allowing  many  repetitions  of  the  same 
experiment.  This  allows  statistics  such  as  the  average  and  mean  square 
error  (MSE)  to  be  calculated  very  easily. 

A.  PROGRAM 

In  the  simulation,  the  exponential  failure  times  were  calculated 
stochastically  using  a  transform  on  the  output  of  a  uniform  pseudo-random 
number  generator.  Times  for  each  cause  were  generated  and  then  the 
minimum  time  was  selected  as  the  next  failure  and  its  cause  was  recorded 
as  the  cause  of  the  failure.  This  is  similar  to  the  "fixed  phase 
reliability  option"  used  by  Chandler  [ref.  5].  Rather  than  generating  a 
failure  time  for  each  item  on  test,  the  time  until  the  first  of  n  items 
fails  was  generated.  This  time  is  the  minimum  of  n  exponential  random 
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variables  with  failure  rate  A.  The  program  keeps  track  of  total  test  time 
in  each  phase,  the  total  time  since  the  last  failure  for  each  cause,  and 
the  adjusted  as  well  as  actual  number  of  failures  for  each  cause  in  each 
phase.  These  values  are  utilized  to  estimate  the  failure  rate  for  each 
phase  using  the  four  models  discussed  in  the  previous  chapters,  the 
cumulative  AMSAA,  Modified  AMSAA,  Modified  AMSAA  with  regression  weights, 
and  Modified  AMSAA  with  failure  discounting.  The  program  calculates  the 
average  and  MSE  for  the  estimate  of  each  model  over  a  number  of 
replications.  Five  hundred  replications  were  used  for  all  cases  simulated 
in  this  thesis.  Appendix  II  gives  a  more  detailed  description  of  the 
program  and  its  use. 

B.  FAILURE  RATE  PATTERNS 

Eight  patterns  of  reliability  growth  and  non-growth  are  simulated  to 
evaluate  the  four  models  over  a  range  of  possible  cases.  The  Eight 
failure  rate  patterns  simulated  are  taken  from  the  eight  reliability 
patterns  used  by  Chandler  [ref.  5:  pp.  19-30].  These  reliabilities  are 
converted  into  failure  rates  assuming  a  one  hour  nominal  mission  time. 
This  assumption  leads  to  the  relationship  R  =  e'^  or  A  =  -InR.  In  this 
way,  Chandler's  matrices  of  Reliability  were  converted  to  matrices  of 
Failure  rates  in  failures  per  hour.  These  eight  matrices  were  used  as 
inputs  to  the  simulation  and  are  shown  in  Tables  II  through  IX. 

The  failure  rate  patterns  are  depicted  in  Figures  1  through  8.  In  all 
of  the  following  descriptions,  reliability  growth  is  analogous  to  a 
decrease  in  the  failure  rate.  Figure  1  depicts  a  pattern  of  non-concave 
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reliability  growth  which  may  not  be  unusual  in  situations  where  the  exact 
method  or  technology  required  to  correct  a  failure  causing  defect  is  not 
immediately  available,  but,  as  the  systems  evolves  and  the  personnel 
become  more  familiar  with  it,  the  failure  correction  process  proceeds  more 
efficiently.  This  pattern  is  convex  in  the  reliability  function,  e'^^ 

Figure  2  represents  a  pattern  of  increasing,  then  decreasing,  then 
finally  increasing  reliability.  This  type  of  pattern  can  result  from 
experimental  systems  where  the  results  of  design  changes  may  introduce  new 
modes  of  failure  when  they  are  implemented. 

Figure  3  represents  a  pattern  of  growth  which  stagnates  for  a  few 
phases  before  finally  achieving  mature  reliability. 

Figures  4  and  5  depict  conventional  reliability  growth  patterns  one 
would  expect  to  encounter  when  evaluating  the  majority  of  systems. 
Pattern  4  ultimately  attains  a  higher  reliability  (lower  failure  rate) 
than  pattern  5. 

Figures  6,7,  and  8  represent  constant  system  reliabilities  that  are 
moderately  high  medium  and  low  respectively. 

These  eight  patterns  were  simulated  over  500  replications  for  each  of 
the  four  models,  each  of  the  four  discounting  parameters  and  each  of  the 
four  weighting  schemes. 
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Figure  2:  Reliability  Increasing  -  Decreasing  -  Increasing 


Table  III:  PATTERN  2  USER  INPUTS 
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Table  IV:  PATTERN  3  USER  INPUTS 


Phase  Fai lure  Rate 


Cause 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1 

.1508 

.0726 

.0305 

.0305 

.0202 

.0101 

.0101 

.0101 

2 

.1508 

.0726 

.0305 

.0202 

.0101 

.0101 

.0101 

3 

.1508 

.0726 

.0305 

.0305 

.0202 

.0202 

.0101 

.0101 

.0101 

4 

.1508 

.0726 

.0305 

.0305 

.0202 

.0202 

.0101 

.0101 

.0101 

5 

.3011 

.2231 

.1393 

.1393 

.0726 

.0619 

.0619 

.0619 
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Figure  4:  Pattern  4,  Rapid  Increase  to  High  Reliability 


Table  V:  PATTERN  4  USER  INPUTS 


Phase  Fai  lure  Rate 


Cause 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1 

.0202 

.0101 

.0101 

.0050 

.0020 

.0020 

.0020 

.0020 

2 

.0513 

.0202 

.0101 

.0050 

.0020 

.0020 

.0020 

.0020 

3 

.1985 

.0408 

.0101 

.0050 

.0020 

.0020 

.0020 

.0020 

.0020 

.0020 

4 

.2231 

.0408 

.0101 

.0050 

.0020 

.0020 

.0020 

.0020 

.0020 

.0020 

5 

.4155 

.1054 

.0101 

.0050 

.0020 

.0020 

.0020 

.0020 

.0020 

.0020 
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Table  VI:  PATTERN  5  USER  INPUTS 
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Figure  6:  Pattern  6,  Constant  Noderately  High  Reliability 


Table  VII;  PATTERN  6  USER  INPUTS 


Phase  Failure  Rate 


Cause 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1 

.0101 

.0305 

.0202 

2 

.0202 

.0202 

.0253 

.0305 

.0101 

.0101 

.0305 

.0253 

.0202 

3 

.0202 

.0253 

.0305 

.0101 

.0202  • 

.0202 

.0101 

.0305 

.0253 

.0202 

4 

.0253 

.0305 

.0101 

.0202 

.0202 

.0202 

.0202 

.0101 

.0305 

mm 

5 

.0101 

.0202 

.0202 

.0253 

.0253 

.0202 

.0202 

.0101 
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Table  VIII:  PAHERN  7  USER  INPUTS 


Phase  Failure  Rate 


Cause 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

1 

.0101 

.2485 

.1054 

.0408 

.1054 

.2485 

.0101 

2 

.2485 

.1054 

.1054 

.0101 

.0101 

.1054 

.0408 

.1054 

,2485 

3 

.1054 

.0408 

.0101 

.2485 

.2485 

.0101 

.1054 

.0408 

.1054 

4 

.0408 

nijm 

.2485 

.1054 

.1054 

.2485 

.1054 

5 

.1054 

.2485 

.1054 

.0408 

.0408 

.1054 

.2485 

.1054 
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Table  IX:  PAHERN  8  USER  INPUTS 


Phase  Failure  Rate 


Cause 

1 

2 

4 

5 

6 

7 

8 

9 

10 

1 

.0202 

.0513 

.1985 

.2231 

.4155 

.4155 

.2231 

.1985 

.0513 

.0202 

2 

.0513 

.1985 

.2231 

.4155 

.0202 

.0202 

.4155 

.2231 

.1985 

.0513 

3 

.1985 

.2231 

.4155 

.0202 

.0513 

.0513 

.0202 

.4155 

.2231 

.1985 

4 

.2231 

.4155 

.0202 

.0513 

.1985 

.1985 

.0513 

.0202 

.4155 

.2231 

5 

.4155 

.0202 

.0513 

.1985 

.2231 

.2231 

.1985 

.0513 

.0202 

.4155 
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V.  INTERPRETATION  OF  RESULTS 


This  chapter  describes  the  results  of  the  simulation  runs  for  the 
eight  failure  patterns.  Four  runs,  designated  E,  F,  G  and  H,  were 
conducted  for  each  pattern  and  each  model.  The  parameters  used  in  failure 
discounting  and  the  weighting  scheme  used  were  varied  over  these  four 
runs.  The  performance  of  the  four  models  for  these  runs  are  reduced  to 
two  graphs,  for  each  pattern  and  each  run,  to  allow  comparison.  The  first 
graph  plots  the  average  of  the  estimates  of  over  the  500  replications 
and  the  true  failure  rate  against  the  phase.  The  second  graph  plots  the 
Mean  square  error  (MSE)  for  the  estimates  against  the  phase.  The  MSE  is 
used  to  compare  the  performance  of  the  models  relative  to  each  other.  The 
MSE  is  calculated  for  each  model  and  each  phase  by 

MSE  =  2(VTk-\)V500  (5.1) 

where  the  summation  is  over  the  500  replications.  Due  to  space 
constraints,  the  graphical  results  of  only  a  few  of  the  more 
representative  runs  are  included  in  this  chapter.  The  graphs  for  all  runs 
conducted  are  included  in  Appendix  C. 

This  interpretation  is  organized  according  to  the  reliability  growth 
patterns  introduced  in  the  previous  chapter.  These  patterns  will  be 
referred  to  by  their  numerical  designator.  The  numerals  are  listed  on  the 
figures  in  the  preceding  chapter  and  are  summarized  below: 

•  Pattern  1  -  Convexly  increasing  reliability 

•  Pattern  2  -  Reliability  increasing  rapidly,  then  decreasing  then 
increasing  again 
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•  Pattern  3  -  Reliability  increasing  rapidly,  then  constant,  then 
increasing  again 

•  Pattern  4  -  Rapidly  increasing  to  high  reliability 

•  Pattern  5  -  Rapidly  increasing  to  moderately  high  reliability 

•  Pattern  6  -  Constant  moderately  high  reliability 

•  Pattern  7  -  Constant  moderate  reliability 

•  Pattern  8  -  Constant  low  reliability 


Recall  that  in  all  of  the  following  graphs  the  standard  AMSAA  model 
was  not  discounted  nor  weighted.  Each  of  these  Patterns  was  evaluated  for 
four  sets  of  regression  weights  and  four  sets  of  discounting  parameters 
as  summarized  in  Tables  X  and  XI  respectively. 


Table  X:  REGRESSION  WEIGHTS  USED 


Run  123456789  10 

E  .050  .224  .368  .473  .549  .607  .652  .688  .717  .741 

F  .250  .500  .620  .707  .758  .794  .820  .841  .857  .871 

G  .500  .707  .794  .841  .871  .891  .905  .917  .925  .933 

H  .100  .100  .110  .120  .130  .140  .150  .700  .800  .900 

Notes: 

1.  E,  F,  and  G  used  weighting  method  1,  w,*  =  f^^',  f  =Wj*. 

2.  H  use^  weighting  method  2,  user  selected  weights. 

3.  All  Wj  above  are  divided  by  the  sum  of  weights  when  used  in  the 
regression  equations  to  ensure  that  weights  always  sum  to  one. 


Table  XI:  DISCOUNTING  PARAMETERS  USED 


Time  Required 

Run  Fraction  Between  Discounting 


E 

.50 

3 

F 

.50 

15 

G 

.25 

3 

H 

.25 

15 
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A.  RAPIDLY  INCREASING  RELIABILITIES:  PAHERNS  4  AND  5 


Figure  9  shows  the  average  estimates  of  the  four  models  applied  to 


Avg  Est .  Fa i lure  Rate 

PQ'^T0r"n  S  E  CUM  AMSAA 

o  MOD  AMSAA 


Pnase 


Figure  9:  Pattern  5  E 

pattern  5.  Patterns  of  this  type  are  considered  typical  reliability 
growth  patterns.  All  four  models  follow  this  pattern  well.  The  Modified 
AMSAA  model  is  closest  to  the  true  failure  rate  in  the  earlier  phases,  and 
the  Cumulative  AMSAA  model  gives  a  slightly  closer  estimate  in  the  last 
three  phases. 


Figure  10:  Pattern  5  E  MSE  (Logarithmic  scale  on  vertical  axis) 

Figure  10  depicts  the  MSE  of  the  estimates  in  Figure  9  (notice  the 
logarithmic  vertical  scale.)  The  Modified  AMSAA  model  has  the  lowest  MSE 
in  the  early  phases,  and  the  Cumulative  AMSAA  has  lower  MSE  in  the  last 
three  phases.  It  is  interesting  to  note  how  the  Modified  AMSAA  model  with 
failure  discounting  has  the  highest  MSE  in  the  later  phases.  Due  to  the 
aggressive  discounting  done  on  this  run,  the  estimates  become  too 
optimistic  and  the  failure  rate  is  underestimated. 


B.  CONVEXLY  INCREASING  RELIABILITY:  PAHERN  1 

Figure  11  indicates  the  performance  of  the  models  on  a  nonstandard 
pattern  of  reliability  growth.  The  Cumulative  AMSAA  model  does  not  follow 
this  pattern.  The  Modified  AMSAA  models  perform  the  best  on  this  type  of 
pattern. 
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As  Figure  12  indicates,  the  Modified  AMSAA  with  failure  discounting 
has  the  lowest  MSE  in  the  last  four  phases.  However  this  model  appears 
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Figure  12;  Pattern  1  E  MSE  (Logarithmic  vertical  scale) 


to  be  biased  toward  the  optimistic  side.  Figure  11  shows  its  tendency  to 
underestimate  the  failure  rate. 


C.  COMBINED  RELIABILITY  GROWTH  AND  NONGROWTH:  PATTERNS  2  AND  3 

Figure  13  shows  the  average  performance  of  these  models  on  Pattern  2. 
The  Modified  AMSAA  model  with  regression  weights  appears  to  be  most 
responsive  to  these  direction  changes.  However,  Figure  14  indicates  that 


the  MSE  of  this  weighted  model  is  not  smallest.  This  is  due  to  the 
increased  variability  of  this  model.  Figure  14  also  shows  that  the 
discounted  model  has  the  lowest  MSE  in  the  last  four  phases,  although  it 
consistently  underestimates  the  failure  rate. 
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MSE 


The  unembellished  Modified  AMSAA  model  performs  well  in  both  average 
and  MSE  at  all  phases  of  this  pattern.  Though  the  Modified  AMSAA  model 
has  a  slightly  higher  MSE  in  the  last  stages,  it  is  better  suited  to 


N/ean  Squared  Error 

Pattern  2  E  amsaa 

O  MOD  AMSAA 


Phase 


Figure  14:  Pattern  2  E  MSE  (Logarithmic  Vertical  scale) 


models  of  this  sort  because  it  is  not  as  optimistic  as  the  discounted 
version. 


D.  PATTERNS  OF  CONSTANT  RELIABILITY:  PAHERNS  6,  7  AND  8 

Figure  15  depicts  the  average  of  the  estimates  when  applied  to 
Pattern  7.  After  the  first  two  phases,  the  Modified  AMSAA  model  follows 
the  constant  reliability  pattern  well.  Figure  16  also  shows  that  this 
model  has  the  lowest  MSE  for  constant  failure  rates. 
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Figure 

15:  Pattern  7  E 

The  preceding  examples  are  representative  of  the  results  for  all 
other  patterns  and  runs.  The  graphical  results  of  all  simulation  runs 


conducted  with  these  models  can  be  found  in  Appendix  C. 

The  Modified  AMSAA  model  without  discounting  or  weighting  showed  the 
best  overall  performance  on  all  patterns  simulated.  There  may  be  certain 
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Figure  16:  Pattern  7  E  MSE  (Logarithmic  vertical  scale) 
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reliability  patterns  where,  given  the  correct  parameters,  the  discounted 
or  weighted  versions  of  this  model  will  yield  a  lower  MSE.  The  reality 
is  that,  in  practice,  the  actual  failure  rate  is  never  known  with 
certainty.  This  fact,  coupled  with  the  difficulty  in  selecting  'good' 
weighting  parameters  and  computational  simplicity,  appear  to  make  the 
Modified  AMSAA  model  a  very  good  choice  for  all  applications. 
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VI.  SUMMARY,  CONCLUSIONS  AND  RECOMMENDATIONS 


A.  SUMMARY 

The  purpose  of  this  paper  was  to  evaluate  the  Modified  AMSAA  model  to 
discover  if  failure  discounting  or  regression  weighting  factors  can 
improve  this  model.  Discounting  failures  occurring  early  in  a  testing 
program  should  allow  the  early  data  to  be  used  to  evaluate  current 
estimates  of  failure  rate.  If  this  discounting  is  warranted  and  is 
applied  correctly,  more  accurate  estimates  of  failure  rate  and  ultimately 
reliability  should  result.  Similarly  the  application  of  weighting  factors 
was  intended  to  improve  the  accuracy  of  the  model  by  forcing  more  weight 
on  the  current  phase  data  in  the  regression. 

Toward  this  end,  two  methods  of  weighting  the  data  were  derived  and 
evaluated.  The  first  method  allows  the  user  to  select  the  first  phase 
weight  and  the  model  computes  each  subsequent  phases  weight.  The  other 
method  allows  the  user  to  select  the  weight  explicitly  for  each  phase. 
Normalized  weights  are  used  in  the  regression  equations. 

The  discounting  method  removed  a  certain  fraction  of  each  failure 
attributed  to  a  given  cause  each  time  the  required  amount  of  test  time  was 
accumulated  without  that  cause  reoccurring.  If  another  failure  occurred 
due  to  that  cause,  the  time  since  failure  was  reset  to  zero  and  all 
failures  due  to  that  cause  were  restored  to  full  value.  This  method  of 
discounting  requires  the  selection  of  two  parameters.  These  are  the 
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fraction  to  be  removed,  f,  and  the  time  required  before  removal,  Treq. 


B.  CONCLUSIONS 

The  Modified  AMSAA  model  was  discovered  to  be  superior  to  the  AMSAA 
model  regardless  of  the  underlying  pattern  of  reliability.  The 
application  of  regression  weighting  factors  or  failure  discounting  did  not 
significantly  improve  the  accuracy  of  this  model.  With  most  patterns 
simulated  these  embellishments  actually  detracted  from  the  accuracy  of  the 
Modified  AMSAA  model.  The  discounted  model  had  a  tendency  to 
underestimate  the  failure  rate.  This  is  an  indication  that  the 
discounting  is  unwarranted  and  probably  not  needed  for  this  model. 

The  reason  for  this  result  could  be  that  the  Modified  AMSAA  model 
uses  regression  methods  to  estimate  the  failure  rate  from  the  data  from 
each  phase.  This  data  is  kept  segregated  by  phase  throughout.  There  may 
be  no  need  to  discount  early  failures  if  this  early  data  is  already 
handled  differently  than  current  data  by  the  model. 

The  weighting  methods  evaluated  failed  to  improve  the  Modified  AMSAA 
model.  There  may  be  a  method  which  optimally  weights  the  failure  data 
from  the  previous  and  current  phases  which  can  improve  this  model.  The 
weighting  methods  applied  in  this  study  were  not  exhaustive.  More 
analysis  should  be  done  to  find  better  methods  of  weighting  this  model. 

The  Modified  AMSAA  model  is  very  easy  to  apply  to  virtually  any  test 
plan.  The  calculations  required  could  be  done  on  most  hand-held 
programmable  calculators.  The  subroutine  located  in  Appendix  B,  could  be 
used  by  anyone  using  this  model  on  a  computer  that  will  support  Fortran. 
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C.  RECOMMENDATIONS  FOR  FURTHER  STUDY 

The  following  is  a  list  of  areas  which  suggest  further  study. 

•  Some  method  optimally  weighting  the  data  should  be  sought  so  that 
the  regression  generated  estimates  have  the  lowest  variance.  A 
study  of  many  possible  weighting  methods  should  be  conducted. 
Ultimately  a  method  of  calculating  the  weights  entirely  from  the 
data  would  be  best. 


•  A  sensitivity  study  should  be  conducted  on  the  choice  of  failure 
discounting  parameters  in  hopes  of  finding  a  way  to  optimally 
discount  failures.  Possibly  a  method  where  these  parameters  are 
calculated  dynamically  from  the  data  could  be  found. 


•  Some  exploration  of  other  methods  for  improving  the  Modified  AMSAA 
model  should  be  studied.  These  would  include  applying  jack-knifing 
to  the  regression  model. 
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APPENDIX  A 


A.  DERIVATION  ESTIMATE  OF  SINGLE  PHASE  FAILURE  RATE  ESTIMATE 

The  Modified  AMSAA  model  assumes  an  exponential  failure  distribution 
with  the  failure  rate  constant  within  each  phase  of  development.  The 
failure  rate  changes  after  the  system  configuration  is  changed,  marking 
the  transition  to  a  new  phase.  Let  denote  the  failure  rate  in  phase  i. 
In  the  test  plan  simulated  in  this  thesis,  testing  is  conducted  on  n  items 
in  each  phase  until  a  set  number  of  failures,  F,  have  been  observed. 
After  the  testing  is  completed,  failures  are  attributed  to  specific  causes 
and  changes  are  made  in  the  design  attempting  to  eliminate  these  causes. 

The  regression  equations  of  the  Modified  AMSAA  model  require  an 
estimate  of  single  phase  failure  rate,  X,.  This  estimate  is  used  in  the 
regression  equations  to  estimate  the  parameters  a  and  b  in  the  model  of 
the  instantaneous  failure  rate, 

Xttj  -  (l-a)b(TT^)'*  [ref.  1:  p.  3-1]. 

In  this  thesis  the  test  plan  simulated  tested  in  each  phase  until  a 
predetermined  number  of  failures  had  occurred.  The  maximum  likelihood 
estimate  for 

-  ^  A  Fj 

Xj  *  — j - ,  has  expected  value  E[Xj]  =  -f — p-  •  X^,  if  Fj  >  1. 

To  calculate  this  expected  value  note  that  ZX^T^  has  a  Chi-square 
distribution  with  parameter  2Fj.  If  X  =  2XT  is  Chi -square(2F) ,  then 
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E[l/X]  =  1/[2(F-1)].  Using  this  estimate  in  the  model  produced  biased 
results.  The  estimated  instantaneous  failure  rate  was  consistently 
high. 

The  next  estimate  tried  was  the  bias  corrected  maximum  likelihood 
estimate, 

A  Fj-1  Fj 

Xj  =  - = -  •  -j - ,  which  has  expected  value  E[Xj]  =  X^. 

j 

Using  this  estimate  the  model  also  produced  biased  results.  The  estimated 
instantaneous  failure  rate  Xjjj  was  consistently  low.  Runs  A  through  D 
used  this  estimate.  The  results  are  not  included  in  this  thesis  because 
they  were  unsatisfactory. 

These  results  prompted  the  search  for  a  nearly  unbiased  estimate  of 
single  phase  failure  rate  that  would  give  unbiased  results  in  the  model. 
The  nearly  unbiased  estimate  chosen  was 


A  2Fj-l 

X  =  — - — 

J  F. 


2F,-1 


■j - ,  which  has  expected  value  E[Xj]  =  — 


if  Fj  >  2.  This  estimate  produced  the  results  shown  in  this  thesis  for 
runs  E  through  G.  The  variance  of  this  estimate  can  be  shown  to  be 


Var[ij)  .  4(Fj-i)'(f,-2)  •  .  'f  Fj  >2. 


The  calculation  of  this  variance  also  uses  the  Chi-square  distribution  of 


2XJj.  If  X  -  2XT  is  Chi-square{2F)  then  E[l/X^]  «  l/[4(F-l)(F-2)],  and 
Var[l/X']  =  E[l/X']  -  {E[l/X])'  -  l/[4{F-l)2{F-2)] . 
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B.  DERIVATION  OF  THE  HODIFIED  AHSAA  MODEL 

The  Modified  AMSAA  model  uses  exponential  regression  to  estimate  the 
parameters  a  and  b  which  relate  the  amount  of  accumulated  test  time,  TT 
to  the  failure  rate.  The  model  is 

=  (l-a)b(TTj)-^  [ref.  1:  p.  3-1]. 

This  relationship  is  transformed  in  order  to  use  the  least  squares  linear 
regression  equations  to  estimate  a  and  b.  If  the  natural  log  is  taken  of 
both  sides  of  this  equation,  the  result  is 

In(XTTj)  =  ln{(l-a)b}  +  (-a)ln(TTj)  =  o  +  ^{ln{TTj)),  if  we  let 
a  =  ln{(l-a)b}  and  -  -a.  Thus  the  data  pairs  become  (In(Xj) ,ln(TTj)) . 
Applying  the  standard  regression  equations  the  estimates  for  a  and  b 
become 

1  X^Y,  -  Yj  2  X, 

3  _ i=i _ 

^  j  j  2 

Xj  2  X,  -  2  X,* 

i=l  i=l 

bj  =  -4—  exp(Yj+  4^) 

where  Y,  =  ln(XJ  and  X  =  ln(TT^),Yj  =  {Yj+Y2+. .  .+Yp/j  and  Xj  = 

{Xj+X2+. . .+Xj)/j . 
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APPENDIX  B 


The  computer  program  written  for  this  thesis  is  written  in  modular 
form.  The  program  is  written  in  six  sections;  the  main  program,  the 
uniform  pseudo-random  number  generator,  and  the  four  subroutines,  one  for 
each  reliability  growth  model.  The  main  program  reads  the  data  file, 
CRG.DAT,  and  uses  the  parameters  contained  in  it  to  control  the 
simulation.  The  simulation  consists  of  three  distinct  steps.  These  are 
generation  of  the  random  variables,  processing  these  random  variables  into 
the  failure  data  needed  by  the  subroutines,  and  finally  use  of  the  data 
by  the  four  subroutines  to  estimate  failure  rates.  The  program  also 
collects  data  on  the  estimates  generated  in  order  to  generate  statistics. 
These  statistics,  the  average  and  mean  square  error  are  used  evaluate 
performance  of  the  models  with  respect  to  the  "known"  reliability  growth 
pattern  used  to  generate  the  data. 

The  subroutines  could  be  used  for  any  program  that  read  the 
appropriate  data  and  used  to  track  reliability  growth  for  an  actual 
system  that  the  user  was  interested  in.  It  could  also  be  used  to  evaluate 
any  continuous  reliability  growth  model  provided  the  user  programs  a 
subroutine  in  such  a  way  that  it  interfaces  with  the  main  program. 

The  names  of  the  variables  and  arrays  used  in  the  main  program  and 
subroutines  have  been  used  so  that  they  are  similar  to  those  parameters 
and  variables  in  the  models  presented  in  the  body  of  this  thesis. 
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The  remainder  of  this  appendix  is  an  explanation  of  variables  and 
arrays  used  in  the  main  program,  a  listing  of  the  source  code  for  the 
program  and  subroutines,  and  sample  input  and  output  files. 


A.  ARRAY  AND  VARIABLE  LIST 


Name 
NCAUSE 
NPHASE 
NREPS 
NITEMS(  ) 

R(  ) 

FCAUSE(  ,  ) 
CAUSE 
PHASE 
J 


Description 

The  number  of  failure  causes  in  the  simulation 

The  number  of  phases  to  be  simulated 

The  number  of  replications  to  simulate  over 

The  number  of  items  on  test  in  each  phase,  indexed  by  phase 

The  number  of  failures  per  phase,  indexed  by  phase 

The  cause  of  a  given  failure,  indexed  by  phase  and  failure 

Index  for  failure  causes 

Index  for  phases  of  development 

Index  for  failures  within  a  phase 


F{  ,  )  The  number  of  failures  for  a  gi^n  cause  in  each  phase, 

indexed  by  phase  and  cause 

FAIL  Counter  of  total  number  of  failures  over  all  phases  and 

causes,  used  to  index  FTT(  ),  for  the  Cumulative  AMSAA  model 


FTT(  ) 
WTP 


T(  ,  ,  ) 


TT(  ,  ) 


Total  test  time  accumulated  by  all  items  after  each  failure, 
indexed  by  failure 

Weighting  type  pointer  -  Data  file  parameter  to  select 
weighting  method  (1  or  2) 

Exponential  failure  time  for  the  first  failure  of  n  items  for 
a  given  cause  in  each  phase,  indexed  by  phase,  cause,  and 
failure 

The  time  on  test  for  all  items  until  the  next  failure  occurs 
for  any  cause,  indexed  by  phase  and  failure 


42 


PTT(  ) 
TSF(  ,  ) 
LAM(  ,  ) 
TRLAM(  ) 
LAMBRU  ) 
LAMBR2(  ) 

LAMBR3(  ) 

LAMBR4(  ) 

MSE1(  ) 

MSE2(  ) 

MSE3{  ) 

MSE4(  ) 

FADJ(  ) 
SEED 


Total  test  time  accumulated  by  all  items  for  each  phase, 
indexed  by  phase 

Test  time  accumulated  since  last  failure  for  a  given  cause 
at  the  end  of  each  phase,  indexed  by  phase  and  cause 

The  failure  rate  (lambda)  for  a  given  cause  in  each  phase, 
indexed  by  phase  and  cause 

True  lambda  -  the  sum  over  all  causes  of  LAM(  ,  )  for  each 
phase,  indexed  by  phase 

The  average  of  the  Modified  AMSAA  model  (MODEL  1)  estimate 
of  lambda  for  each  phase,  indexed  by  phase 

The  average  over  all  replications  of  the  Weighted  Modified 
AMSAA  model  (MODEL  2)  estimate  of  lambda  for  each  phase, 
indexed  by  phase 

The  average  over  all  replications  of  the  Discounted  Modified 
AMSAA  model  (MODEL  3)  estimate  of  lambda  for  each  phase, 
indexed  by  phase 

The  average  over  all  replications  of  the  Cumulative  AMSAA 
model  (MODEL  4)  estimate  of  lambda  for  each  phase,  indexed 
by  phase 

The  mean  square  error  over  all  replications  of  the  Modified 
AMSAA  model  (MODEL  1)  estimate  of  lambda  for  each  phase, 
indexed  by  phase 

The  mean  square  error  over  all  replications  of  the  Weighted 
Modified  AMSAA  model  (MODEL  2)  estimate  of  lambda  for  each 
phase,  indexed  by  phase 


The  mean  square  error  over  all  replications  of  the  Discounted 
Modified  AMSAA  model  (MODEL  3)  estimate  of  lambda  for  each 
phase,  indexed  by  phase 

The  mean  square  error  over  all  replications  of  the  Cumulative 
AMSAA  model  (MODEL  1)  estimate  of  lambda  for  each  phase, 
indexed  by  phase 


The  adjusted  number  of  failures  for  a  given  cause  at  the  end 
of  each  phase,  indexed  by  phase  and  cause 

The  seed  for  the  random  number  generator  function  RAND(SEED) 
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B.  LISTING  OF  COMPUTER  PROGRAM  SOURCE  CODE:  CRG.FOR 


***************************************************************** 
*  * 

*  Fortran  program  to  Simulate  Continuous  failure  data  from  * 

*  failure  rates  for  specific  causes.  The  data  is  used  to  test  * 

*  four  Reliability  Growth  Models  and  evaluate  results  by  * 

*  comparing  Mean  Square  Error.  * 

*  This  program  will  handle  up  to  15  phases,  10  failure  causes,  * 

*  and  10  failures  per  phase.  If  needed  the  dimensions  on  the  * 

*  arrays  could  be  changed  to  accommodate  more.  * 


*  * 

*  * 

*  * 

*  PROGRAMMED  BY  :  * 

*  LT  S.L. NEGUS  USN  * 

*  * 


***************************************************************** 


INTEGER  NCAUSE,NPHASE,NREPS,NITEMS(15),R{15),FCAUSE(15,10), 

9  CAUSE, PHASE, F(15, 10), FAIL, WTP 

DOUBLE  PRECISION  T(15,10,10),TT{15,10),PTT(15),TSF{0:15,10) 

@  ,LAM(15,10),LAMBRI(15),LAMBR2(15),LAMBR3{15),LAMBR4(15) 

@  ,MSE1(15),MSE2{15),MSE3(15),MSE4(15),FADJ(15,15),SEED 

9  ,FTT{0:150),TRLAM{15) 

OPEN  (UNIT=1,FILE='CRG.DAT') 

OPEN  {UNIT=9,FILE='MSES.OUT') 

OPEN  (UNITES, FILE»'AVGS. OUT') 

*  TAKE  DATA  FROM  DATA  FILE  AND  INITIALIZE  VARIABLES 

*  READ  IN  #  CAUSES,  §  PHASES,  #  REPLICATIONS,  SEED  &  WEIGHTING  *  TYPE 
POINTER  FROM  DATA  FILE 

READ ( 1 , *)  NCAUSE , NPHASE , NREPS, SEED, WTP 

*  READ  IN  FAILURE  DISCOUNTING  FRACTION  AND  TIME  INTERVAL 

READ(1,*)  FRAC,TREQ 

*  READ  IN  FAILURE  RATE  FOR  EACH  PHASE  AND  CAUSE 

READ{ I , * )  ( ( LAM( PHASE , CAUSE) , PHASE- 1 , NPHASE) , CAUSE- 1 , NCAUSE ) 

*  READ  IN  #  ITEMS  &  #  FAILURES  PER  PHASE  FROM  DATA  FILE 

DO  1  PHASE  =  1, NPHASE 

READ  (1,*)  NITEMS(PHASE),R{PHASE) 

DO  2  CAUSE  -  1. NCAUSE 

TRLAM(PHASE)  -  TRLAM{PHASE)+LAM(PHASE, CAUSE) 

2  CONTINUE 

1  CONTINUE 
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*  MAIN  DO  LOOP  TO  CARRY  OUT  PROCEDURE  NREPS  TIMES 

DO  100  Nl, NREPS 
FAIL  =  0 
FTT(O)  -  0. 

*  SIMULATE  THE  TIME  TO  FAILURE  FOR  THE  JTH  FAILURE  IN  EACH  PHASE 

*  AND  FOR  EACH  CAUSE 

DO  10  PHASE  -  1,  NPHASE 
DO  15  CAUSE  .  1,  NCAUSE 
TSF( PHASE, CAUSE) -0.0 
DO  17  J-1,R(PHASE) 

T(PHASE, CAUSE, J)-(-l./((NITEMS(PHASE)-J+l)* 

,  ®  LAM(PHASE,CAUSE)))*AL0G(RAND(SEED)) 

17  CONTINUE  ^  \  II 

15  CONTINUE 

10  CONTINUE 

*  SORT  THE  TIMES  FOR  EACH  CAUSE  IN  EACH  PHASE  &  FAILURE  TO 

*  DETERMINE  WHICH  CAUSE  FAILED  FIRST  AND  WHEN  THAT  HAPPENED 

DO  20  PHASE  =  1, NPHASE 
PTT( PHASE) =0.0 
DO  25  J  =  1,R(PHASE) 

FCAUSE(PHASE,J)  -  0 
TT(PHASE,J)  =  1.0E25 
DO  30  CAUSE  «  1, NCAUSE 

IF  (T(PHASE, CAUSE, J)  .LT.  TT(PHASE,J))  THEN 
TT(PHASE,J)  =  T( PHASE, CAUSE, J) 

FCAUSE(PHASE,J)  =  CAUSE 
ENDIF 

30  CONTINUE 


CALCULATE  THE  TIME  SINCE  LAST  FAILURE  FOR  EACH  CAUSE  IN  PHASE 

DO  40  CAUSE  -  1, NCAUSE 

IF  (CAUSE  .EQ.  FCAUSE(PHASE,J))  THEN 
TSF( PHASE, CAUSE)  -  0.0 
ELSEIF  (J  .EQ.  1)  THEN 

TSF(PHASE, CAUSE)  -  TSF ( PHASE -1, CAUSE )+ 

TT(PHASE,J)*(NITEMS(PHASE)-J+1) 


ELSE 


TSF(PHASE, CAUSE)  -  TSF{PHASE,CAUSE)+ 

TT(PHASE,J)*(NITEMS(PHASE)-J+1) 


40  CONTINUE 

'  COMPUTE  TOTAL  TIME  ON  TEST  FOR  EACH  PHASE,  PTT(PHASE) 
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AND  FOR  EACH  FAILURE,  FTT{FAIL) 


* 


PTT ( PHAS  E ) = PTT ( PHAS  E ) +TT ( PHAS  E , J ) * ( N I T  EMS ( PHAS  E ) - J + 1 ) 
FAIL  =  FAIL  +  1 

FTT(FAIL)=FTT{FAIL-1)+TT(PHASE,J)*(NITEMS{PHASE)-J+1) 
25  CONTINUE 

*  ADJUST  FAILURES  FOR  FAILURE  DISCOUNTING 

DO  75  K=l, PHASE 
FADJ( PHASE, K)  =  0 
75  CONTINUE 

DO  50  CAUSE  -  1,NCAUSE 
F(PHASE,CAUSE)=0 
DO  60  J-1,R(PHASE) 

IF  (CAUSE  .EQ.  FCAUSE(PHASE,J))  THEN 
F(PHASE, CAUSE)  =  F( PHASE, CAUSE)  +  1 
END  IF 

60  CONTINUE 

DO  70  K=l, PHASE 

FADJ(PHASE,K)=FADJ(PHASE,K)+F(K, CAUSE) 

0  *(!.-  FRAC)**AINT(TSF(PHASE,CAUSE)/TREQ) 

70  CONTINUE 

50  CONTINUE 

20  CONTINUE 

*  GO  TO  SUBROUTINES  FOR  MODIFIED  AMSAA  MODELS 

CALL  MODL I ( I , PTT , R , NPHASE , LAMBRl , MSEl , TRLAM) 

CALL  M0DL2 ( WTP , I , PTT , R , NPHASE , LAMBR2 , MSE2 , TRLAM) 

CALL  M0DL3 ( I , PTT , FADJ .NPHASE , LAMBR3 , MSE3 , TRLAM) 

CALL  M0DL4 ( I , FTT , R , NPHASE , LAMBR4 , MSE4 , TRLAM , FA I L ) 

100  CONTINUE 

*  FORMAT  AND  WRITE  OUTPUT  TO  FILE 

WRITE  (5,140)  NREPS 

WRITE  (9,145)  NREPS 

DO  120  PHASE-1, NPHASE 

LAMBRl (PHASE ) -LAMBRl (PHASE )/NREPS 
MSEl ( PHASE ) -MSE 1 ( PHASE )/NREPS 
LAMBR2 ( PHASE) -LAMBR2 ( PHASE)/NREPS 
MSE2 ( PHASE) -MSE2 ( PHASE )/NREPS 
LAMBR3 ( PHASE ) -LAMBR3 ( PHASE)/NREPS 
MSE3 ( PHASE ) -MSE3 ( PHASE )/NREPS 
LAMBR4 ( PHASE) -LAMBR4 ( PHASE)/NREPS 
MSE4(PHASE)=MSE4(PHASE)/NREPS 

WRITE  (5,150)  TRLAM(PHASE), LAMBRl (PHASE), LAMBR2(PHASE) 

0  , LAMBR3 ( PHASE ),LAMBR4( PHASE) 
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WRITE  (9,150)  MSEl (PHASE), MSE2(PHASE),MSE3(PHASE) 
@  ,MSE4(PHASE) 


120  CONTINUE 

140  FORMAT  ('AVERAGE  OF  ESTIMATES  FOR', 16,'  REPS'/ 

0  '  TRUE  LAMBDA  MODEL  1  EST  MODEL  2  EST  MODEL  3  EST', 

0  '  MODEL  4  EST') 

145  FORMAT  ('MEAN  SQUARE  ERRORS  FOR', 16,'  REPS'/ 

0  '  MODEL  1  MSE  MODEL  2  MSE  MODEL  3  MSE  MODEL  4  MSE') 

150  FORMAT  (6F13.8) 

STOP 

END 

*  RANDOM  NUMBER  GENERATOR 

FUNCTION  RAND(SEED) 

DOUBLE  PRECISION  M,A,X,SEED 
M=2.**31-l. 

/^_7  **5 

X=l!  +  MOD(A*SEED,M) 

RAND=X/M 

SEED=X 

RETURN 

END 
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it  it 

*  SUBROUTINE  #1  * 

*  * 

*  FORTRAN  MODEL  FOR  RELIABILITY  IN  CONTINUOUS  PROCESS  * 

*  PROGRAMMED  BY  :  * 

*  LT  S.L. NEGUS  USN  * 

*  * 

*  THIS  MODEL  IS  BASED  ON  THE  MODIFIED  AMSAA  MODEL.  * 

*  USING  LEAST  SQUARES  REGRESSION.  * 

*  THE  CONTINUOUS  FAILURE  RATE  IS  ESTIMATED.  * 

*  NO  DISCOUNTING  OR  REGRESSION  WEIGHTS  ARE  USED.  * 

*  * 


***************************************************************** 


*  SUBROUTINE  PROGRAM  TO  ESTIMATE  LAMBDA  USING  MODIFIED  AMSAA  MODEL 
SUBROUT I N E  MODL 1 ( I , T , F , NPHASE , LAMBAR , MS E , LAM) 

INTEGER  F{15) 

DOUBLE  PRECISION  T(15) ,TT,X,Y,SUMXY,SUMY,XBAR, 

0  YBAR , SUMX , SUMX2 , AHAT , BHAT , LAMHAT 
0  ,LAMBAR(15),MSE(15),LAM(I5) 


itit'kitititititicit’kitititititilitititititititititititititititititititititititititieitit 


*  MAIN  DO  LOOP  * 

*  * 

♦  ITERATIONS  ON  DEVELOPMENT  PHASE,  K  ^ 

*  ★ 


itititititititititititititiciticiiititititititiiititltitititkitititititititititititicitit 


TT=0. 

SUMXY=0. 

SUMY=0. 

SUMX=0. 

SUMX2=0. 


DO  200  K-1, NPHASE 

IF  (F(K)  .LE.  1)  THEN 

Y  -  ALOG(,5/T(K)) 

ELSE 

Y  »  AL0G((2.0*F{K)-1.0)/(2.0*T(K))) 
ENDIF 

TT-TT+T(K) 

X=ALOG(TT) 

SUMXY=SUMXY+X*Y 

SUMY=SUMY+Y 


SUMX*SUMX+X 

SUMX2-SUMX2+X*X 

XBAR-SUMX/K 

YBAR*SUMY/K 

IF  (K  .GT.  1)  THEN 

AHAT  » ( SUMX Y - YBAR*SUMX )/ ( XBAR*SUMX 
0  -SUMX2) 

BHAT=(1 ./{I . -AHAT))*EXP{YBAR+AHAT*XBAR) 
LAMHAT- ( 1 - AHAT ) *BHAT*TT** ( -AHAT ) 

ELSEIF  (F(K)  .GT.  1)  THEN 

LAMHAT-{2.0*F{K)-1.0)/{2.0*T(K)) 

ELSE 

LAMHAT..5/T(K) 

ENDIF 

LAMBAR { K ) - LAMBAR ( K ) + L AMHAT 
MSE(K)  =  MSE(K)  +  (LAMHAT-LAM(K) )**2 


200  CONTINUE 

210  F0RMAT(3F6.1) 

220  F0RMAT(I3,5F10.4) 

230  FORMAT ( A3, 5A10) 

END 
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It**************************************************************** 


*  SUBROUTINE  #2  * 

*  * 

*  FORTRAN  MODEL  FOR  RELIABILITY  IN  CONTINUOUS  PROCESS  * 

*  PROGRAMMED  BY  :  * 

*  LT  S.L. NEGUS  USN  * 

*  * 

*  THIS  MODEL  IS  BASED  ON  THE  MODIFIED  AMSAA  MODEL  * 

*  USING  WEIGHTED  LEAST  SQUARES  REGRESSION.  * 

*  THE  CONTINUOUS  FAILURE  RATE  IS  ESTIMATED.  * 

*  REGRESSION  WEIGHTS  ARE  USED.  * 

*  * 


*  SUBROUTINE  PROGRAM  TO  ESTIMATE  LAMBDA  USING  MODIFIED  AMSAA  MODEL 
SUBROUT  I NE  MOOL2  ( WTP ,  IJ ,  F ,  NPHASE ,  LAMBAR ,  MSE ,  LAM ) 

INTEGER  F(15),N{15),WTP 

DOUBLE  PRECISION  T(15) ,TT(0: 15) ,X(15) , Y{15) ,SUMXY{15) 

@  ,SUMY(0: 15) ,XBAR(15) , YBAR(15) ,SUMX(0; 15) ,SUMX2{15) ,AHAT(15) 

I?  ,BHAT(15),LAMBAR{15),MSE(15),LAM(15),W(15),SUMW{0:15),SUMLAM 
@  ,LAMHAT(15) 

IF  (I  .EQ.  1)  OPEN  (UNIT»3,FILE«'WEIGHTS.DAT') 

■k-kitir’kiftcit'k’k'ttit'kitlt'kit'k-k'kit'k-kit'kititititifkifit'kit'kicitiiit'k'kitlt'k 


*  MAIN  DO  LOOP  * 

*  * 

*  ITERATIONS  ON  DEVELOPMENT  PHASE,  K  * 

*  * 


Itleititleificititicltitisitigititicititigititititic’kigieititiriciciciflcit'k'kifit'k'kie 

TT(0)»0 

SUMY{0)=0 

SUMX(0)-0 

SUMW(0)>0 


SUMLAM  -  0. 

DO  300  K-1, NPHASE 


*  FIND  THE  WEIGHT  FOR  THIS  PHASE 

IF  (WTP  .EQ.  1)  THEN 
IF  (I  .EQ.  1)  THEN 

*  READ  IN  WEIGHT 

READ(3,*)  W(K) 
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ENDIF 

ELSE 

*  COMPUTE  WEIGHTS  USING  FRAC**(1/K)  FORMULA 

IF  ((I  .EQ.  1).AND.(K  .EQ.  1))  THEN 

*  READ  IN  FRACTION 

READ{3,*)  FRAC 
ENDIF 

W(K)  =  FRAC**(1./K) 

ENDIF 

SUMW(K)=SUMW{K-1)+W(K) 

IF  (F(K)  .LE.  1)  THEN 
Y(K)-ALOG(.5/T(J)) 

ELSE 

Y(K)-AL0G((2.0*F(K)-1.0)/(2.0*T(K))) 

ENDIF 

TT(K)=TT(K-1)+T(K) 

X(K)=ALOG(TT(K)) 

SUMY{K)-SUMY(K-1)+Y(K)*W(K) 

SUMX(K)-SUMX(K-1)+X(K)*W(K) 

XBAR{K)=SUMX(K)/SUMW(K) 

YBAR(K)=SUMY(K)/SUMW(K) 

SUMXY(K)  =  0. 

SUMX2(K)  =  0. 

DO  350  J=1,K 

SUMXY(K)»SUMXY(K)+W(J)*{X(J)-XBAR(K))*Y{J) 
SUMX2(K)*SUMX2(K)+W(J)*(X(J)-XBAR(K))**2 
350  CONTINUE 


IF  (K  .GT.  1)  THEN 

AHAT(K).(-1.0)*SUMXY(K)/SUMX2(K) 

BHAT(K)=(1./(1.-AHAT(K)))*EXP(YBAR(K)+AHAT(K)*XBAR(K)) 
L AMHAT ( K ) = ( 1 - AHAT ( K ) ) *BHAT (K)*TT(K)**(-AHAT(K)) 

ELSEIF  (F(K)  .GT.  1)  ^MEN 

LAMHAT(K)-(2.0*F(K)-1.0)/(2.0*T(K)) 

ELSE 

LAMHAT(K)=.5/T(K) 

ENDIF 

LAMBAR(K)»LAMBAR{K)+LAMHAT(K) 

MSE{K)  -  MSE{K)  +  (LAMHAT(K) -LAM{K) )**2 

300  CONTINUE 

310  F0RMAT{3F6.1) 

320  F0RMAT(I2,5F10.4) 

330  F0RMAT(A2,5A10) 

END 
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***************************************************************** 


SUBROUTINE  #3 

FORTRAN  MODEL  FOR  RELIABILITY  IN  CONTINUOUS  PROCESS 
PROGRAMMED  BY  : 

LT  S.L. NEGUS  USN 

THIS  MODEL  IS  BASED  ON  THE  MODIFIED  AMSAA  MODEL. 
USING  LEAST  SQUARES  REGRESSION. 

THE  CONTINUOUS  FAILURE  RATE  IS  ESTIMATED. 

FAILURE  DISCOUNTING  IS  USED. 


*  SUBROUTINE  PROGRAM  TO  ESTIMATE  LAMBDA  USING  MODIFIED  AMSAA  MODEL 
SUBROUTINE  MODL3 ( I , T , F , NPHASE , LAMBAR , MSE , LAM) 

INTEGER  N{15) 

DOUBLE  PRECISION  T{15) ,TT(0: 15) ,X(15) , Y(15) ,SUMXY(15) , 

@  SUMY(I5),XBAR{I5),YBAR(15),SUMX(15),SUMX2(15),AHAT(15), 

@  BHAT(15),LAMBAR(I5),MSE(15),LAM(15),F(15,15),LAMHAT(15) 


hicititititicitieititisieieieieitifificititifieit’kieitie'kif'kii'kicifificititititititie 


*  MAIN  DO  LOOP  * 

*  * 

*  ITERATIONS  ON  DEVELOPMENT  PHASE,  K  * 

★  ★ 


It'k'kie'kicicitirir'kitit'kitieitiririr-kieieie’kie'k'kititicitiriririf’kif'kitif’k’kifk 


TT(0)=0 

DO  200  K=l, NPHASE 
SUMXY(K)-') 
SUMY{K)=0 
SUMX(K)=0 
SUMX2(K)=0 
TT{K)=TT(K-1)+T(K) 


DO  250  J*1,K 

IF  (F(K,J)  .LE.  1.5)  THEN 
Y(J)=AL0G{.5/T(J)) 

ELSE 

Y(K)=ALOG((2.0*F(K,J)-1.0)/(2.0*T(J))) 

ENDIF 

X(K)=ALOG(TT{K)) 

SUMXY(K)=SUMXY(K)+X(J)*Y(J) 
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SUMY(K)=SUMY(K)+Y{J) 

SUMX(K)»SUMX(K)+X(J) 

SUMX2(K)-SUMX2(K)+X(J)*X(J) 

250  CONTINUE 

XBAR(K)»SUMX(K)/K 
YBAR(K)*SUMY(K)/K 
IF  (K  .GT.  1)  THEN 

AHAT{K)-{SUMXY(K)-YBAR(K)*SUMX(K))/(XBAR(K)*SUMX(K) 

@  -SUMX2(K)) 

BHAT(K)-(1./(1--AHAT(K)))*EXP(YBAR{K)+AHAT(K)*XBAR(K)) 

LAMHAT(K)-(1-AHAT(K))*BHAT(K)*TT(K)**(-AHAT(K)) 

ELSE  IF  (F(K,K)  .GT.  1.)  THEN 

LAMHAT{K)-{2.0*F(K,K)-1.0)/(2.0*T(K)) 

ELSE 

LAMHAT(K)-  .5  /  T(K) 

ENDIF 

L AMBAR { K ) = LAMBAR { K ) + L AMHAT { K ) 

MSE(K)  =  MSE(K)  +  (LAMHAT{K) -LAM(K) )**2 

200  CONTINUE 

210  F0RMAT(3F6.1) 

220  FORMAT(I2,5F10.4) 

230  FORMAT (A2,5A10) 

END 
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itififititifk’k'kititititicit'kif'kieicicifkifk'k'k’kifitit'kitifk'kicititicifkieitificititieicitit'kitif'ieiflfkicirii’k’kir 


★  * 

*  SUBROUTINE  #4  * 

*  * 

*  FORTRAN  MODEL  FOR  RELIABILITY  IN  CONTINUOUS  PROCESS  * 


*  PROGRAMMED  BY  :  * 

*  LT  S.L. NEGUS  USN  * 

*  * 

*  THIS  MODEL  IS  BASED  ON  THE  AMSAA  MODEL.  * 

*  THE  CONTINUOUS  FAILURE  RATE  IS  ESTIMATED.  * 

*  * 

*  * 


*  SUBROUTINE  PROGRAM  TO  ESTIMATE  LAMBDA  USING  AMSAA  MODEL 
SUBROUTINE  MODL4 ( I , X , R , NPHASE , LAMBAR , MSE , TRLAM , NFA I L ) 
INTEGER  R(I5), NPHASE, NFAIL, I, J,K,N 

DOUBLE  PRECISION  X(0;I50) ,LAMBAR{15) ,MSE{15) ,TRLAM(15) ,SUM 
0  ,AHAT(15),BHAT(15),LAMHAT{15) 


N=0 

DO  no  K=I, NPHASE 
N=N+R(K) 

SUM  =  0.0 

IF  (N  .GT.  2)  THEN 

*  CALCULATE  SUMMATION  FOR  ESTIMATES 

DO  100  0=1, N-1 

IF  (X{N)/X(J).GT.O.)  THEN 
SUM  =  SUM  +  ALOG(X(N)/X(J)) 

ELSE 

PRINT  *, 'ERROR  IN  SUBROUTINE  4;  NEG  TIME  AT  REP  ',I 
STOP 
ENDIF 

100  CONTINUE 

*  CALCULATE  ESTIMATES  OF  PARAMETERS  AND  FAILURE  RATE 

IF  (SUM.NE.O.)  THEN 
AHAT(K)  -  N/SUM 
ELSE 

PRINT  *, 'ERROR  IN  SUBROUTINE  4;  BAD  SUM  REP  ',I 
STOP 
ENDIF 

BHAT{K)  =  N/{X(N)**AHAT(K)) 

LAMHAT(K)  =  AHAT(K)*BHAT{K)*X(N)**(AHAT(K) -1 .0) 

ELSE 

LAMHAT  (K)  =  N/X(N) 

ENDIF 

LAMBAR(K)  =  LAMBAR(K)  +  LAMHAT (K) 
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MSE{K)  =  MSE{K)  +{LAMHAT{K)-TRLAM(K))**2 
110  CONTINUE 

*  ERROR  CHECK:  IS  FINAL  N  =  NFAIL? 

IF  (N  .NE.  NFAIL)  THEN 

PRINT  *, 'ERROR  IN  SUBROUTINE  4;  DISAGREEMENT  IN 
?  #  FAILURES', I 
STOP 
ENDIF 

END 


C.  SAMPLE  USER  DATA  FILE:  CRG.DAT 

5,10,500,12345.,!  NUMBER  OF  CAUSES,  PHASES,  REPS,  SEED,  WT  SCHEME 
.5,3.  DISCOUNTING  FRACTION,  TIME  INTERVAL 


.0101 

.2485 

.1054 

.0408 

.1054 

.1054  .0408 

.1054 

.2485 

.0101 

.2485 

.1054 

.0408 

.1054 

.0101 

.0101  .1054 

.0408 

.1054 

.2485 

.1054 

.0408 

.1054 

.0101 

.2485 

.2485  .0101 

.1054 

.0408 

.1054 

.0408 

.1054 

.0101 

.2485 

.1054 

.1054  .2485 

.0101 

.1054 

.0408 

.1054 

.0101 

.2485 

.1054 

.0408 

.0408  .1054 

.2485 

.0101 

.1054 

5,2 

NUMBER 

OF  ITEMS 

AND 

FAILURES 

IN 

PHASE  1 

5,2 

NUMBER 

OF  ITEMS 

AND 

FAILURES 

IN 

PHASE  2 

5,2 

NUMBER 

OF  ITEMS 

AND 

FAILURES 

IN 

PHASE  3 

5,2 

NUMBER 

OF  ITEMS 

AND 

FAILURES 

IN 

PHASE  4 

5,2 

NUMBER 

OF  ITEMS 

AND 

FAILURES 

IN 

PHASE  5 

5,2 

NUMBER 

OF  ITEMS 

AND 

FAILURES 

IN 

PHASE  6 

5,2 

NUMBER 

OF  ITEMS 

AND 

FAILURES 

IN 

PHASE  7 

5,2 

NUMBER 

OF  ITEMS 

AND 

FAILURES 

IN 

PHASE  8 

5,2 

NUMBER 

OF  ITEMS 

AND 

FAILURES 

IN 

PHASE  9 

5,2 

NUMBER 

OF  ITEMS 

AND 

FAILURES 

IN 

PHASE  10 
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0.  SAMPLE  OUTPUT  FILES:  AVGS.OUT  AND  MSES.OUT 


1.  AVGS.OUT 


AVERAGE  OF  ESTIMATES  FOR  500  REPS 


TRUE  LAMBDA 

MODEL  1  EST 

MODEL  2  EST 

MODEL  3  EST 

.51020000 

.71328117 

.71328117 

.70052859 

.51020000 

.76562159 

.76562159 

.74092911 

.51020000 

.56986163 

.57546390 

.52658969 

.51020000 

.54309383 

.54768820 

.44106557 

.51020000 

.50977294 

.51543531 

.35891453 

.51020000 

.49092691 

.49542132 

.31900548 

.51020000 

.48351571 

.48732093 

.28569272 

.51020000 

.47925587 

.55408753 

.25654757 

.51020000 

.46936838 

.50655576 

.23365031 

.51020000 

.47131996 

.50018151 

.22722248 

2.  MSES.OUT 


MEAN  SQUARE  ERRORS  FOR  500  REPS 


MODEL  1  MSE 

MODEL  2  MSE 

MODEL  3  MSE 

MODEL  4  MSE 

2.28307676 

2.28307676 

2.29055386 

4.17982537 

1.59749806 

1.59749806 

1.61366470 

8.03775319 

.24587519 

.26528818 

.30665339 

.67145094 

.18450393 

.18848388 

.25013123 

.35297324 

.09295514 

.09939919 

.12774927 

.19431540 

.07139884 

.07607480 

.10168462 

.13055892 

.05210290 

.05293369 

.09214781 

.09232873 

.04455537 

.21215101 

.09099763 

.07024485 

.03619510 

.07225663 

.09299745 

.05374019 

.03441349 

.05769722 

.09465450 

.04438188 

MODEL  4  EST 
.95104156 
1.50951350 
.93848540 
.81369015 
.72585658 
.67593184 
.64586001 
.62174413 
.59856596 
.58609344 
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APPENDIX  C 


This  appendix  contains  the  graphical  presentation  of  the  results  from 
all  simulation  runs  conducted  with  the  models  presented  and  developed  in 
this  thesis.  The  graphs  are  identical  in  format  to  those  presented  in 
Chapter  V.  For  completeness,  those  graphs  presented  in  chapter  5  will  be 
repeated  here  with  all  the  other  graphs. 

These  graphs  are  organized  according  to  the  runs  and  the  reliability 
growth  patterns  within  those  runs.  Each  of  these  Patterns  was  evaluated 
for  four  sets  of  regression  weights  and  four  sets  of  discounting 
parameters  as  summarized  in  tables  XII  and  XIII  respectively. 

Table  XII:  REGRESSION  WEIGHTS  USED 


Run  123456789  !0 

E  .050  .224  .368  .473  .549  .607  .652  .688  .717  .741 

F  .250  .500  .620  .707  .758  .794  .820  .841  .857  .871 

G  .500  .707  .794  .841  .871  .891  .905  .917  .925  .933 

H  .100  .100  .110  .120  .130  .140  .150  .700  .800  .900 

Notes: 

1.  E,  F,  and  G  used  weighting  method  1,  w,*  =  f  =Wj*. 

2.  H  use<J  weighting  method  2,  user  selected  weights. 

3.  All  w^  above  are  divided  by  the  sum  of  weights  when  used  in  the 
regression  equations  to  ensure  that  weights  always  sum  to  one. 
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Table  XIII:  DISCOUNTING  PARAMETERS  USED 


Time  Required 

Run  Fraction  Between  Discounting 


E 

.50 

3 

F 

.50 

15 

G 

.25 

3 

H 

.25 

15 

The  patterns  are  those  introduced  in  Chapter  IV.  These  patterns  are 
referred  to  by  the  numerical  designator  .  The  numerals  are  listed  on  the 
figures  and  are  summarized  below: 

•  Pattern  1  -  Convexly  increasing  reliability 

•  Pattern  2  -  Reliability  increasing  rapidly,  then  decreasing  then 
increasing  again 

•  Pattern  3  -  Reliability  increasing  rapidly,  then  constant,  then 
increasing  again 

•  Pattern  4  -  Rapidly  increasing  to  high  reliability 

•  Pattern  5  -  Rapidly  increasing  to  moderately  high  reliability 

•  Pattern  6  -  Constant  moderately  high  reliability 

•  Pattern  7  -  Constant  moderate  reliability 

•  Pattern  8  -  Constant  low  reliability 

Once  again  increasing  reliability  is  synonymous  with  decreasing  failure 
rate. 
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